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Abstract: There is continued debate about the changing and varied nature of first year college 
courses in American higher education institutions; therefore, additional research is needed to 
inform faculty and administration of the specific areas on which to focus for teaching and learning 
in these courses. This study is one possible approach to easily determine student “gains’’ as 
measured by the College Success Factors Index, or CSFI. This index consists often college student 
success factors that indicate student achievement. The purpose of this research was to examine 
the CSFI pretest and post-test data for possible student gains. These self- evaluations were 
administered in both the summer 2013 and summer 2014 semester courses. The CSFI pre and 
post-test scores comparing each summer were analyzed and the overall gains were observed. The 
summer 2013 courses were taught without a focus on the targeted success factors. A curricular 
intervention was then introduced in the summer 2014 courses that integrated the ten success 
indices into the lessons, activities, and assignments. The overall differences or “gains ” in student 
learning are described, along with implications for first year teaching and program design in 
colleges and universities. 
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Introduction 

The design and subsequent continuous improvement of a first year experience course that 
maximizes student learning and success is a very complex endeavor. Depending on the changing 
priorities or administration, level of student aptitude, faculty skill/experience level and a host of 
other factors, first year experience courses have a tendency to be placed in a certain category. The 
2009 National Survey of First Year Seminars (Padgett and Keup, 2011) has identified five common 
types: (a) extended orientation, (b) academic with uniform content, (c) academic on various topics, 
(d) pre-professional or discipline linked, and (e) basic study skills. 

The first year experience course that was examined in this study is a life skills/study skills 
college success course offered at the University of Central Florida with the course number and title 
of SLS1501: Strategies for College Student Success. This class followed the traditional approach 
of presenting the learning topics and issues based on research about first-year college student 
achievement. The teaching of these important topics is not based on opinion or simple advice, but 
rather rigorous research related to college student success; therefore, the course is an academic 
workshop with uniform content. 

In an effort to improve, innovate, and assess the effectiveness of the summer of 2013 
workshop, the steering committee for the course required faculty in all sections to administer the 
College Success Factors Index, or CSFI. The CSFI will be described in greater detail throughout 
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the ensuing section. The CSFI was administered at the beginning of the course and again at the 
end, measuring the self-reported gains. The committee met again in the fall of 2013 to discuss the 
results and decided that a curricular intervention that includes the ten CSFI student success factors 
would increase the probability of student achievement. During the spring of 2014, the uniform 
curriculum was redesigned, and by the summer, the new curriculum was implemented. These 
indices were fully integrated into the lectures, assignments, activities, and research projects, and 
then later measured. The steering committee was very interested in any “gains” or improvements 
that teaching the ten factors would yield in the summer 2014 course as the factors were not a direct 
focus in the summer 2013 curriculum. Thus, the idea for this study took shape and will hopefully 
inform other programs and institutions about how to approach curriculum design with the objective 
of improving and promoting first-year student learning. 

Literature Review 

In an effort to better understand and address the many challenges that first-year college 
students’ face which hinder their ability to be successful and eventually persist to graduation, 
Cengage Learning developed an assessment tool entitled the College Student Factors Index, or the 
CSFI. This self-reporting instrument is now being utilized in select first year experience courses 
and has three primary objectives: (1) to identify “at-risk” students via early-alert reporting, (2) put 
focused intervention strategies into place, and (3) address the comprehensive development of 
college students via an in-depth analysis of their social, emotional, and cognitive factors (2015, 
May 11). 

The CSFI centers on ten student success factors grounded in research on the college 
student, which are predictive of students’ long-term ability to succeed in college. It has a 
coefficient alpha of .91 and many validity studies have been conducted resulting in coefficients of 
-.30 to -.50, which is desirable for self-reporting instruments. This assessment tool is administered 
online and consists of a pretest and post-test. The pretest is given at the beginning of the course, 
quickly identifying students’ strengths and opportunities for growth and then provides the students, 
along with their respective instructors, with a detailed evaluation, student profile, and 
recommendations for precise intervention strategies. The post-test is later used to measure student 
gains in these areas. The ten student success factors that make up the CSFI are linked to thoroughly 
researched areas within higher education regarding student development and persistence to 
graduation. 

Success Factors 

“Responsibility/Control” over assignments and other accountabilities are associated with 
attribution theory. There are two psychological constructs related to this first student success 
factor: locus of control and motivation. Locus of control consists of two facets: internal locus, 
where students attribute their successes and failures in college to their own efforts and motivations, 
and external locus, believing their outcomes are due to chance. According to Bean and Eaton 
(2001), students with an internal locus of control connect their academic and social achievements 
to studying and attending classes which bolsters their motivation, while others with an external 
locus believe that their achievements are related to luck or professors ‘liking’ them. Bean and 
Eaton (2001) stated that student efforts and psychological responses to their interactions with the 
college lead to academic and social integration. The external locus of control mindset can severely 
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limit college students’ motivation to take responsibility for their academic and social 
achievements, thus limiting their chances of integration, which plays a fundamental role in 
retention and persistence to graduation. 

“Competition”, between other college students or internally, make up the second student 
success factor. This is related to academic and social integration. “The more students are involved 
in the social and academic life of the institution, the more likely they are to leam and persist” 
(Tinto, 1998, p. 2). Students need to forge relationships with fellow classmates and faculty and 
engage in academic discourse. According to Zhao and Kuh (2004), an excellent way to build 
academic and social integration is to have college students participate in learning communities; 
the results from their research indicate that learning communities are linked to greater academic 
effort, better academic integration, more contact with faculty, and higher order thinking skills. It 
is essential that we challenge students intellectually and help motivate them to excel in their 
studies. 

“Task Planning”, or the ability to break down tasks or projects to ensure they are 
completed, is linked to organizational and study skills. Goldfinch and Hughes (2007) stated that 
study skills are correlated with academic performance and emphasized long-term planning as a 
key aspect of first-year college student success. It is essential for students to break down tasks as 
there will be many assignments, and this level of organization will assist students in finishing their 
coursework by meeting stringent deadlines. 

“Expectations”, in this particular context, are connected to goal setting and career planning. 
These goals must be specific and can really improve the likelihood of success and raise students’ 
self-efficacy. Schunk (1990) specified that goals must integrate specific performance criteria, 
which may enhance learning and the probability of meeting the specified goal. Goals are 
connected to a broad number of motivational variables that set the stage for positive achievements 
(Ames, 1992). Working toward a specific goal will improve students’ self-efficacy, and according 
to Bandura and Cervone (1986), self-efficacy notably influences the amount of effort expended, 
and ultimately, persistence toward the goal. 

“Family Involvement” is the participation and encouragement provided by family members 
of college students in assisting them with career planning and decision-making (2015, May 11). 
According to McCarron and Inkelas (2006), the involvement of parents in first-generation college 
students lives has a significant impact on their educational aspirations. Playing such a powerful 
role, it is suggested that higher education professionals working in the area of recruitment reach 
out to parents directly through orientation, open houses, and high school bridge programs. 

“College Involvement” describes the amount of interaction that students have with their 
institutions. Astin (1999) provided a student involvement theory and asserts that involvement is 
defined as “the amount of physical or psychological energy that a student devotes to the academic 
experience” (p. 518). The premise of his theory is that students highly involved are more likely to 
engage in positive activities such as studying, interacting with faculty, joining organizations, 
and/or spending more time on campus. Conversely, uninvolved students may not devote enough 
time to their studies, have less interaction with faculty, and abstain from extracurricular activities. 

“Time Management” refers to college students’ ability to organize and maximize their time 
and to rank the high priority objectives from the lower priority tasks. To be successful in college, 
it is important that students’ manage their time wisely and efficiently. Macon, Shahani, Dipboye, 
and Phillips (1990) found that students who self-identified as good time managers reported greater 
performance, work and life satisfaction, and less somatic tensions. Not only does efficient time 
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management help ensure academic success, it aids in relieving stress associated with poor time 
management and/or procrastination. 

“Wellness” is described as the emotional and physical health of college students. In college, 
students are faced with a variety of obstacles and challenges, which may produce stress, 
sleeplessness, and inadequate diets. Pritchard and Wilson (2003) examined the impact of college 
students’ emotional and physical health, specifically depression and fatigue on GPA and retention. 
It was found that students who reported they intend to drop out were more fatigued and had lower 
self-esteem. Conversely, students who were planning on staying enrolled developed coping 
mechanisms to deal with taxing situations that may arise. 

“Precision” is aligned with personality type, or a particular mentality. Our mindset and 
personal preferences can go a long way in how we approach certain situations and face new 
challenges. Individuals have preferences about how they like to learn information, interact with 
the material, and complete tasks. “Students with different type preferences tend to respond 
differently to different modes of instruction” (Felder, Felder, & Dietz, 2002, p. 3). It is very 
important that college students are cognizant of their personality types, attitudes, and preferences 
to ensure a proper plan of action is taken when tackling challenges related to college. 

“Persistence” is linked to self-efficacy as it is defined as facing a task with self¬ 
encouragement, diligence, and personal urgency. It is how students feel about approaching their 
goals, objectives, and assignments. Bandura (1997) described the notion of self-efficacy as “the 
belief in one’s capabilities to organize and execute courses of action required to produce given 
attainments” (p. 3). Self-efficacy is having confidence in your ability to complete a particular task. 
Chemers, Hu, and Garcia (2001) guided a longitudinal study measuring the effects of self-efficacy 
and optimism on students’ academic performance, which were strongly related to adjustment to 
the college experience. 

The ten student success factors that make up the CFSI have been outlined with selected 
higher education research corresponding to and supporting these facets. This brief snapshot of 
facets stresses the qualities necessary for college students to be successful. 

Methodology 


Purpose Statement 

Pretest/post-test designs are widely used in behavioral research, primarily for the purpose 
of comparing groups and/or measuring change resulting from experimental treatments or 
interventions. The purpose of this study is to measure the gains in academic performance as 
measured across the ten CSFI factors of student success. The measurement of that change provides 
a vehicle for assessing the effectiveness of the intervention (teaching/curriculum). 

Research Questions 

1. Are there gains in student performance from the pre-test and post-test in the summer 2013 and 
summer 2014 for the SLS1501: Strategies for Student Success course? 

2. Are there significant differences in the gains in student performance from the post-tests between 
summer 2013 and summer 2014? 
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Participants and Setting 

The participants in the study were first-year college students enrolled in SLS1501: Strategies for 
College Student Success at the University of Central Florida in the summer of 2013 (N=431) and 
the summer 2014 (N=593). No additional demographic data were collected for the purposes of this 
study. The course is a 3 credit hour graded course and is 6 weeks in length. 

Intervention 

The CSFI was administered during the first week of class and the last week of class in each 
of the summer 2013 and summer 2014 course sections. Figure 1 below represents the pre/post 
assessment model that reflects the intervention of new curriculum in summer 2014. Recall that for 
the summer 2013 course, there was a uniform curricular approach to the ten CSFI college success 
factors. 

Figure 1 below illustrates the intervention which consisted of a complete overhaul of the 
syllabus to include the 10 success factors that are measured in the CSFI. This means that the 10 
success factors were intentionally and directly taught using a variety of instructional strategies that 
varied from lectures, activities, exercises, assignments, readings, and research on the topic. 



Figure 1. Pre and Post Assessment Model. This represents the curricular intervention 
implemented in summer 2014. 
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Data Collection 

Data were collected during the summer 2013 and summer 2014 sections of the first year 
experience course, SLS1501: Strategies for College Student Success. The College Success Factors 
Index (CSFI) was administered at the beginning and the end of each course. The instrument was 
completed online and the data were automatically compiled by course section and the university 
as a whole. The university data (all students who completed the CSFI) comprised the data analyzed 
in this study. 

The CSFI presents college students with a series of statements that measure their ability to 
be successful in college by asking them to complete a Likert style self-reporting survey with five 
choices that range from “Strongly Agree” to “Strongly Disagree”. It includes the ten CSFI college 
success factors previously described in the literature review. The instrument accounts for 
approximately ten questions centering on each of the ten college success factors, for a total of 100 
items. The score range for each factor is from 10 (high success) to 50 (low success). A high score 
reflects negative performance and conversely, a low score reflects a high possibility of success. 
The lowest overall score is 100 and the highest possible score is 500. 

Data Analysis 

To detennine if student performance improved “within” each semester independently, 
descriptive statistics including means and standard deviations for each of the semesters were 
calculated. To conclude that the differences in scores were significant, specifically that an increase 
in performance or “gains” existed among the summer 2013 and the summer 2014 sections, 
independent t-tests were conducted. 

Findings 

Table 1 below provides the means and standard deviations of the College Success Factors 
Index (CSFI) pretest and post-test at the beginning and end of the summer 2013 and summer 2014 
semester course of first-time incoming freshmen. It is important to remember that the scale used 
for CSFI is comprised of ten success indices: Responsibility & Control; Competition; Task 
Planning; Expectations; Wellness; Time Management; College Involvement; Family Involvement, 
Precision, and Persistence, and that on this particular scale, a decrease in scores suggests an 
increase in student learning. 

Table 1 also shows the results of the 2013 CSFI pretest (mean= 19.684, Std= 3.9161), the 
CSFI post-test (mean= 17.124, Std= 4.1295), the 2014 CSFI pretest (mean= 17.468, Std= 2.4586) 
and the CSFI post-test (mean= 16.548, Std= 3.7619) which illustrates that the means have 
decreased. 
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Table 1 

Means and Standard Deviations 




Mean 

N 

Std. 

Std. Error 





Deviation 

Mean 

Pair 1 

pre 2013 

19.684 

431 

3.9161 

.1886 

post 2013 

17.124 

431 

4.1295 

.1989 

Pair 2 

pre 2014 

17.468 

593 

2.4586 

.1010 

post 2014 

16.548 

593 

3.7619 

.1545 


Table 2 highlights the outcomes of the pre and post-test data of the CSFI scores between 
2013 and 2014, independent t-tests were conducted and Table 2 provides a summary of the results. 

Table 2 

Differences in Gains 





Test Value = 0 




t 

df 

Sig. (2-tailed) 

Mean 

Difference 

95% Confidence Interval of 
the Difference 

Lower Upper 

post 2013 

86.089 

430 

.000 

17.1241 

16.733 

17.515 

post 2014 

91.038 

430 

.000 

16.5227 

16.166 

16.879 


These results demonstrate significant differences between the pre and post-tests of the 
CSFI results between summer 2013 (t-value =86.089, p<0.01) and summer 2014 (t-value =91.038, 
p<0.01). The means and t-values suggest the effectiveness of the curricular intervention which 
translates into gains in student performance. 

Discussion 

The steering committee that directed the administration of the course under investigation 
immediately realized that having students complete the CSFI pre- and post-test in the summer 
2013 was a step in the right direction as collecting and analyzing data about how the students were 
performing in the course provided them with additional insights into student learning. Taking this 
a step further, the committee integrated the ten CSFI success factors into the curriculum and found 
that it provided significant gains in student learning. 

A careful examination of the data reveals that there were improvements in performance in 
summer 2013 as well as summer 2014; however, there was a significant difference in the 
improvements made in summer 2014 than in summer 2013. The curricular intervention showed 
significant gains; though, there are a multitude of other considerations when “standardizing” the 
content occurs. Some important issues that arose were that faculty expressed concerns about 
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unfamiliarity with teaching some of the topics, a possible loss of academic freedom (whatever that 
means), and a lack of autonomy in course content. This is a delicate issue when the goal is to 
improve the course and student performance as well as retain qualified and interested faculty from 
around the university. Ultimately, the steering committee found it necessary to conduct regular 
training sessions and hold meetings to gain the “buy-in” of the faculty. This was important because 
assessing improvements in student performance would be impossible without clear 
communication and consistent training. 

The next step for this program involves continuing to improve the teaching of the success 
factors, the assessment of students across time, and the continued collection of data regarding 
student gains. Padgett and Keup (2010) suggest that what truly facilitates student development is 
the incorporation of vetted good practices into the course. Based on the research regarding the 10 
CSFI success factors, we are well on our way to doing this. We are also progressing toward making 
assessment of the program part of our culture. This is a huge step considering that the 2009 
National Survey of First-Year Programs (Padgett and Keup, 2011) reported that approximately 
one-third of higher education institutions had not conducted an assessment in the previous three 
years. 

This research endeavor does not take into consideration the role of faculty in student 
achievement. The student-professor relationship is essential when considering positive outcomes 
for students as it impacts their quality of learning and increases the confidence that they will do 
well (Micari & Pazos, 2012). Pascarella and Terenzini (1979) also found that informal contact 
between students and professors lead to persistence and retention among first-year college 
students. In addition, Kuh and Hu (2001) stated that the quality of the student-faculty relationship 
has an effect on student learning and their overall satisfaction. In conjunction with the ten CSFI 
success factors, it is necessary to understand that rapport among college students and faculty is 
indispensable. 

Additionally, the revised curriculum focused specifically on the 10 areas that were tested. 
As expected there was be an improvement on the test when the items have been specifically 
addressed in the class. In this instance, it makes sense that the course was revised based on these 
areas of testing. 

For the purposes of this study, only the overall differences or “gains” of the CSFI were 
measured and reported. The aim was to find out if any gains were achieved between pre and post¬ 
test data and then again after the curricular intervention was implemented, thus laying the 
groundwork for future research on this topic. There is now an opportunity to analyze the ten CSFI 
indices individually to understand where the particular gains occurred. 

Conclusion 

The findings of this study suggest that the curriculum of a first year experience course is 
not something to be considered a “last minute thought.” The way a course is taught, the topics to 
be covered, and the assessment of the progress of student learning must be given due diligence and 
adopted in a deliberate way. The results of these efforts will provide significant improvements to 
performance and student learning. It is highly recommended that the ten CSFI success factors are 
integrated into the curriculum of first year experience courses regardless of institutional type, and 
that data regarding student achievement is collected and analyzed to have a better understand of 
how to improve overall gains in learning and performance. 
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